<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.11"/>
<title>Superpixel Benchmark: Benchmark</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
  $(document).ready(function() { init_search(); });
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectalign" style="padding-left: 0.5em;">
   <div id="projectname">Superpixel Benchmark
   </div>
   <div id="projectbrief">Superpixel benchmark, tools and algorithms.</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.11 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
  <div id="navrow1" class="tabs">
    <ul class="tablist">
      <li><a href="index.html"><span>Main&#160;Page</span></a></li>
      <li class="current"><a href="pages.html"><span>Related&#160;Pages</span></a></li>
      <li><a href="annotated.html"><span>Classes</span></a></li>
      <li><a href="files.html"><span>Files</span></a></li>
      <li>
        <div id="MSearchBox" class="MSearchBoxInactive">
        <span class="left">
          <img id="MSearchSelect" src="search/mag_sel.png"
               onmouseover="return searchBox.OnSearchSelectShow()"
               onmouseout="return searchBox.OnSearchSelectHide()"
               alt=""/>
          <input type="text" id="MSearchField" value="Search" accesskey="S"
               onfocus="searchBox.OnSearchFieldFocus(true)" 
               onblur="searchBox.OnSearchFieldFocus(false)" 
               onkeyup="searchBox.OnSearchFieldChange(event)"/>
          </span><span class="right">
            <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
          </span>
        </div>
      </li>
    </ul>
  </div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Benchmark </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>The benchmark combines metrics from different references focussing on different aspects of superpixel segmentations. Detailed equations can be found in the paper or in the <a href="https://davidstutz.github.io/superpixel-benchmark/">Doxygen Documentation</a>, specifically in the class <a href="https://davidstutz.github.io/superpixel-benchmark/classEvaluation.html">Evaluation</a>.</p>
<p>Note: <a class="el" href="classEvaluation.html" title="Provides measures to evaluate (over-) segmentations. ">Evaluation</a> should be done using the executables discussed in <a class="el" href="EXECUTABLES_8md.html">Executables</a>. However, the metrics implemented in the <a href="https://davidstutz.github.io/superpixel-benchmark/classEvaluation.html">Evaluation</a>. class can also be used directly from within C++.</p>
<h2>Boundary Recall</h2>
<p>Boundary Recall is part of the precision-recall framework introduced in [2] and quantifies the fraction of boundary pixels correctly captured by a superpixel segmentation. Higher Boundary Recall describes better adherence to image boundaries. </p><pre class="fragment">[2] D. Martin, C. Fowlkes, J. Malik.
    Learning to detect natural image boundaries using local brightness, color, and texture cues.
    IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (5) (2004) 530–549.
</pre><p>Note: <code><a class="el" href="evaluation_8h.html">lib_eval/evaluation.h</a></code> also includes an implementation of Boundary Precision.</p>
<h2>Undersegmentation Error</h2>
<p>Undersegmentation Error was first introduced in [3] and quantifies the leakage of superpixels across ground truth segments. However, the original formulation penalizes large superpixels (see [3, 4]) covering multiple ground truth segments which may be misleading. Therefore, the formulation of [5] is used. For each superpixel and ground truth segment pair we count the pixels in the overlap or the difference set, whichever is smaller. These pixels constitute the leakage of superpixels across the ground truth segments (assuming that superpixels are assigned to ground truth segments with highest overlap). </p><pre class="fragment">[3] A. Levinshtein, A. Stere, K. N. Kutulakos, D. J. Fleet, S. J. Dickinson, K. Siddiqi.
    TurboPixels: Fast superpixels using geometric flows.
    IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (12) (2009) 2290–2297.
[4] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk.
    SLIC superpixels compared to state-of-the-art superpixel methods.
    IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (11) (2012) 2274–2281.
[5] P. Neubert, P. Protzel. 
    Superpixel benchmark and comparison.
    Forum Bildverarbeitung, 2012.
</pre><p>Note: <code><a class="el" href="evaluation_8h.html">lib_eval/evaluation.h</a></code> includes two different implementations of Undersegmentation Error: the definition by Neubert and Protzel and the definition by Levinshtein.</p>
<h2>Achievable Segmentation Accuracy</h2>
<p>Achievable Segmentation Accuracy [6] quantifies the segmentation performance achievable when using superpixels instead of pixels. To this end, each superpixel is assigned to the ground truth segment with highest overlap. The number of pixels correctly classified this way is used to compute Achievable Segmentation Accuracy. </p><pre class="fragment">[6] M. Y. Lui, O. Tuzel, S. Ramalingam, R. Chellappa.
    Entropy rate superpixel segmentation.
    IEEE Conference on Computer Vision and Pattern Recognition, 2011, pp. 2097–2104.
</pre><h2>Explained Variation</h2>
<p>Explained Variation [7] computes the variance within each superpixels, weights it by the size of the superpixel and normalizes the sum by the total image variance. This way, Explained Variation quantifies the fraction of variance within the image that is captured (i.e. "explained") by the superpixel segmentation. </p><pre class="fragment">[7] A. P. Moore, S. J. D. Prince, J. Warrell, U. Mohammed, G. Jones.
    Superpixel lattices.
    IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
</pre><h2>Compactness</h2>
<p>Compactness [8] compares the area of each superpixel with the area of a circle with the same perimeter. </p><pre class="fragment">[8] A. Schick, M. Fischer, R. Stiefelhagen.
    Measuring and evaluating the compactness of superpixels.
    International Conference on Pattern Recognition, 2012, pp. 930–934.
</pre><p>Note: The implementation in <code><a class="el" href="evaluation_8h.html">lib_eval/evaluation.h</a></code> uses a simple estimate of the perimeter of superpixels. Therefore, results may not be comparable to those in [8].</p>
<h2>Intra-Cluster Variation</h2>
<p>Intra-Cluster Variation [9] computes the average standard deviation of the superpixels. However, this is not done in relation to the overall image variation. </p><pre class="fragment">[9] W. Benesova, M. Kottman.
    Fast superpixel segmentation using morphological processing.
    Conference on Machine Vision and Machine Learning, 2014.
</pre><h2>Mean Distance to Edge</h2>
<p>Mean Distance to Edge [9] averages the distance of each boundary pixel in the ground truth segmentation to the nearest boundary pixel in the superpixel segmentation. Assuming the distances to encode the binary relationship of true positives or false negatives, Mean Distance to Edge resembles Boundary Recall. </p><pre class="fragment">[9] W. Benesova, M. Kottman.
    Fast superpixel segmentation using morphological processing.
    Conference on Machine Vision and Machine Learning, 2014.
</pre><h2>Contour Density</h2>
<p>Contour Density [10] quantifies the fraction of pixels that are boundary pixels in the superpixel segmentation </p><pre class="fragment">[10] V. Machairas, M. Faessel, D. Cardenas-Pena, T. Chabardes, T. Walter, E. Decenciere.
    Waterpixels.
    Transactions on Image Processing 24 (11) (2015) 3707–3716.
</pre><h2>Average Metrics</h2>
<p>In order to render Boundary Recall, Undersegmentation Error and Explained Variation independent of the number of generated superpixels, the area above (or below in the case of Undersegmentation Error) the corresponding curves is used to summarize performance. </p>
</div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Sun Apr 16 2017 16:39:17 for Superpixel Benchmark by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.11
</small></address>
</body>
</html>
